GAO 


United  States  General  Accounting  Office _ 

Report  to  the  Honorable  Dan  Burton, 
Chairman,  Committee  on  Government 
Reform,  House  of  Representatives 


October  2000 


MANAGING  FOR 
RESULTS 


Emerging  Benefits 
From  Selected 
Agencies’  Use  of 
Performance 
Agreements 


°  ApprovSfor” SoBelease 
Distribution  Unlimited 


20001124  151 


GAO 

Accountability  *  Integrity  *  Reliability 


GAO  01-115 


Contents 


Letter 


Results  in  Brief 

Background 

Scope  and  Methodology 

Common  Benefits  Emerge  From  Agencies’  Use  of  Performance 
Agreements 

Concluding  Observations 
Agency  Comments 


Appendixes 


Appendix  I:  VHA’s  Implementation  of 
Performance  Agreements 

Appendix  II:  DOT’s  Implementation  of 
Performance  Agreements 

Appendix  III:  SFA’s  Implementation  of 
Performance  Agreements 


Figures 


Figure  1:  Emerging  Benefits  of  VHA’s,  DOT’s,  and  SFA’s  Use  of 

Results-Oriented  Performance  Agreements  6 

Figure  2:  Translating  DOT  Strategic  Goals  Into  FAA  Programmatic  and 

Executive  Goals  10 

Figure  3:  Excerpt  of  Performance  Information  on  Follow-up  Care 
After  Hospitalization  for  Mental  Illness  in  VHA’s  Quarterly 
Network  Performance  Report  and  VISN  5’s  Biweekly 
Performance  Report  16 

Figure  4:  Excerpts  of  VHA  Performance  Agreement  Template  for  2000  26 


Page  1 


GAO-01  115  Managing  For  Results:  Performance  Agreements 


Abbreviations 


CFIT 

CFO 

CIO 

CNO 

COO 

DOT 

FAA 

GPRA 

MSAW 

NHTSA 

PATS 

PBO 

PI 

RSPA 

SES 

SFA 

VAMHCS 

VHA 

VISN 


Controlled  Flight  into  Terrain 

Chief  Financial  Officer 

Chief  Information  Officer 

Chief  Network  Officer 

Chief  Operating  Officer 

Department  of  Transportation 

Federal  Aviation  Administration 

Government  Performance  and  Results  Act 

Minimum  Safe  Altitude  Warning 

National  Highway  and  Traffic  Safety  Administration 

Performance  Agreement  Tracking  System 

Performance-Based  Organization 

Prevention  Index 

Research  and  Special  Programs  Administration 

Senior  Executive  Service 

Office  of  Student  Financial  Assistance 

Veterans  Affairs  Maryland  Health  Care  System 

Veterans  Health  Administration 

Veterans  Integrated  Service  Network 


Page  2 


GAO-Ol-115  Managing  For  Results:  Performance  Agreements 


i 
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^^^^^^^Accountability  *  Integrity  *  Reliability 


United  States  General  Accounting  Office 
Washington,  D.C.  20548 


October  30,  2000 

The  Honorable  Dan  Burton 

Chairman,  Committee  on  Government  Reform 

House  of  Representatives 

Dear  Mr.  Chairman: 

The  Government  Performance  and  Results  Act  (GPRA)  provides  a  strategic 
management  framework  intended  to  improve  federal  performance  and 
hold  agencies  accountable  for  achieving  results.  Effective  implementation 
of  this  framework  requires  agencies  to  clearly  establish  results-oriented 
performance  goals  for  which  they  will  be  held  accountable,  measure 
progress  towards  those  goals,  determine  the  strategies  and  resources  to 
cost  effectively  accomplish  the  goals,  and  then  use  performance 
information  to  make  the  programmatic  decisions  necessary  to  improve 
performance.  One  way  to  embed  such  a  framework  throughout  agencies  is 
to  align  individual  employee  performance  expectations  with  agency  goals 
so  that  individuals  understand  the  connection  between  their  daily  activities 
and  their  organization’s  success. 

As  a  first  step  towards  this  end,  you  asked  us  to  review  federal  agencies 
that  have  begun  to  use  performance  agreements  to  align  executive 
performance  expectations  with  organizational  goals.  As  agreed,  this  report 
identifies  the  common  emerging  benefits,  if  any,  from  the  Veterans  Health 
Administration’s  (VHA),  the  Department  of  Transportations  (DOT),  and  the 
Office  of  Student  Financial  Assistance’s  (SFA)  implementation  of 
performance  agreements  that  may  be  applicable  to  other  agencies  as  they 
too  seek  to  use  performance  agreements  as  one  tool  to  improve 
organizational  performance  and  accountability.  We  selected  VHA  and  DOT 
because  of  their  experience  with  performance  agreements  since  the  mid- 
1990s,  and  SFA  because  of  its  statutory  requirement  to  use  them. 
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VHA  uses  performance  agreements  with  career  executives  to  help  translate 
its  strategic  goals  into  day-to-day  operations  and  to  hold  those  executives 
accountable  for  achieving  program  results.  VHA’s  performance  agreements 
between  its  Under  Secretary  for  Health  and  each  of  its  22  Veterans 
Integrated  Service  Network1  (VISN)  directors  include  goals  within  each  of 
VHA’s  five  domains  of  health  care-quality,  functional  status,  satisfaction, 
access,  and  cost — as  well  as  other  areas  of  special  interest,  such  as  patient 
safety  The  agreements  also  specify  executive  “core  competencies,”  such  as 
interpersonal  effectiveness,  flexibility/adaptability,  or  technical 
competency,  that  are  considered  during  the  evaluation  process.  VHA 
considers  progress  towards  the  goals  in  the  agreements  when  evaluating 
VISN  director  performance  and  making  performance  reward  decisions. 
(Additional  information  on  VHA’s  implementation  of  performance 
agreements  appears  in  app.  I.) 

DOT  implements  performance  agreements  between  the  Secretary  of 
Transportation  and  the  heads  of  its  eleven  operating  administrations  and 
bureaus  or  “modal  administrators,”  its  Assistant  Secretaries,  and  Office 
Directors.  DOT’S  annual  performance  plan  developed  under  GPRA  serves 
as  the  basis  for  its  performance  agreements  and  DOT  assigns  its  annual 
performance  goals  and  the  activities  intended  to  achieve  those  goals  to  the 
appropriate  executive’s  performance  agreement.  The  agreements  also 
include  “corporate  management  strategies”  that  cut  across  organizational 
boundaries  and  set  additional  expectations  for  administrators,  such  as  to 
use  customer  feedback  to  improve  programs.  These  agreements  represent 
what  modal  administrators  have  agreed  to  accomplish,  but  because 
administrators  are  appointed  by  the  President  and  confirmed  by  the 
Senate,  the  administrators  are  not  eligible  for  cash  awards.  (Additional 
information  on  DOT’s  implementation  of  performance  agreements  appears 
in  app.  II.) 

Additionally,  Congress  has  recognized  the  role  that  performance 
agreements  can  play  in  holding  organizations  and  executives  accountable 
for  results.  In  1998,  Congress  chartered  SFA  as  a  Performance-Based 
Organization  (PBO)  and  required  SFA  to  implement  performance 
agreements.  SFA’s  performance  plan  required  under  its  PBO  legislation  also 
served  as  the  basis  for  the  performance  agreements  at  SFA.  The 
performance  agreement  between  the  Secretary  of  Education  and  SFA’s 


1 A  VISN  is  a  VHA  regional  office  that  has  primary  responsibility  for  health  care  delivery  and 
coordinates  the  activities  of  hospitals,  outpatient  clinics,  nursing  homes,  and  other  facilities. 
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Results  in  Brief 


Chief  Operating  Officer  (COO)  set  three  goals  for  the  COO,  which  are  in  the 
areas  of  customer  satisfaction,  the  cost  of  delivering  student  aid  and 
services,  and  employee  satisfaction.  The  performance  agreements  between 
the  COO  and  the  senior  managers  include  corresponding  goals.  For 
example,  the  General  Manager  for  Schools,  who  is  responsible  for  working 
towards  SFA’s  performance  objectives  related  to  higher  education 
institutions,  is  expected  to  contribute  to  SFA’s  overall  goals,  as  well  as  to 
improve  customer  satisfaction  among  school  administrators  and  financial 
aid  advisors,  reduce  his  or  her  organization’s  unit  costs,  and  improve 
satisfaction  among  employees  who  work  to  support  school  services.  Under 
the  PBO  legislation,  progress  towards  the  goals  in  performance  agreements 
is  to  serve  as  the  basis  by  which  to  determine  the  amount  of  any  annual 
performance  bonuses  for  the  COO  and  the  senior  managers.  (Additional 
information  on  SFA’s  implementation  of  performance  agreements  appears 
in  app.  III.) 


As  part  of  a  series  of  concerted  management  reform  efforts  designed  to 
improve  organizational  performance,  VHA,  DOT,  and  SFA  used  results- 
oriented  performance  agreements  for  their  senior  political  and  career 
executives  to  define  accountability  for  specific  goals,  monitor  progress 
during  the  year,  and  then  contribute  to  performance  evaluations.  Although 
each  agency  developed  and  implemented  agreements  that  reflected  their 
specific  organizational  priorities,  structures,  and  cultures,  we  identified 
five  common  emerging  benefits  from  each  agency’s  use  of  results-oriented 
performance  agreements.  These  emerging  benefits  are  shown  in  figure  1. 
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Figure  1:  Emerging  Benefits  of  VHA’s,  DOT’S,  and  SFA’s  Use  of  Results-Oriented 
Performance  Agreements 


1.  Strengthened  alignment  of  results-oriented  goals  with  daily  operations: 

Performance  agreements  define  accountability  for  specific  goals  and  help 
to  align  daily  operations  with  agencies'  results-oriented,  programmatic  goals. 

2.  Fostered  collaboration  across  organizational  boundaries: 

Performance  agreements  encourage  executives  to  work  across  traditional 
organizational  boundaries  or  "silos"  by  focusing  on  the  achievement  of 
results-oriented  goals. 

3.  Enhanced  opportunities  to  discuss  and  routinely  use  performance 
information  to  make  program  improvements: 

Performance  agreements  facilitate  communication  about  organizational 
performance,  and  provide  opportunities  to  pinpoint  improved  performance. 

4.  Provided  results-oriented  basis  for  individual  accountability: 

Performance  agreements  provide  results-oriented  performance 
information  to  serve  as  the  basis  for  executive  performance  evaluations. 

5.  Maintained  continuity  of  program  goals  during  leadership  transitions: 

Performance  agreements  help  to  maintain  a  consistent  focus  on  a  set  of 
broad  programmatic  priorities  during  changes  in  leadership. 


Source:  GAO  analysis  based  on  VHA’s,  DOT’s)  and  SFA’s  implementation  of  performance  agreements. 

A  shift  in  the  orientation  of  individual  performance  expectations  and 
accountability  from  an  adherence  to  process  or  completion  of  activities  to 
a  greater  focus  on  contributions  to  results  will  require  a  cultural 
transformation  for  most  federal  agencies.  In  this  respect,  officials  at  VHA, 
DOT,  and  SFA  stressed  that  they  are  continually  refining  how  they  develop 
and  then  use  their  agreements.  They  are  also  addressing  ongoing 
implementation  issues,  such  as  refining  their  understanding  of  how  their 
activities  contribute  to  results,  providing  executives  readily  accessible 
performance  information  in  a  useful  timeframe  and  format,  and  further 
cascading  accountability  to  the  appropriate  levels.  As  they — and  the 
federal  government  as  a  whole — gain  experience  with  the  use  of 
performance  agreements,  on  the  basis  of  the  benefits  seen  thus  far,  it 
appears  that  such  agreements  can  become  an  increasingly  vital  part  of 
overall  efforts  to  improve  programmatic  performance  and  better  achieve 
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results.  VHA,  DOT,  and  SFA  generally  agreed  with  the  contents  of  this 
report. 


Background 


A  number  of  the  world’s  major  democracies  have  initiated  performance 
agreements  as  one  element  within  a  broader  series  of  public  sector  reforms 
to  enhance  government  performance  and  strengthen  accountability  for 
results.  For  example,  in  1995,  we  reported  on  four  countries— the  United 
Kingdom,  New  Zealand,  Canada,  and  Australia — that  implemented 
performance  agreements  between  department  leaders  and  their  top  civil 
service  managers  to  instill  a  sense  of  personal  responsibility  for 
performance  and  to  reinforce  the  connection  between  employee 
performance  and  organizational  missions  and  goals.2 

In  the  United  States,  the  federal  government  is  also  seeking  ways  to 
reinforce  the  connection  between  employee  performance  and  agency 
goals.  The  Office  of  Management  and  Budget’s  Circular  A-ll  which 
provides  guidance  on  the  preparation  and  submission  of  agencies’  budget 
requests,  including  GPRA  plans  and  reports,  calls  for  executive  agencies  to 
discuss  in  their  fiscal  year  2002  annual  performance  plans  goals  and 
strategies  to  make  their  workforces  more  performance-oriented,  such  as 
linking  personnel  appraisals  to  program  performance.  In  addition,  a 
February  2000  report  to  the  President’s  Management  Council 
recommended  linking  employee  goals  to  their  agency’s  strategic  goals.  The 
report  highlighted  a  number  of  initiatives,  including  the  use  of  performance 
agreements,  that  can  contribute  to  improving  performance  and  enhancing 
accountability. 

Finally,  the  Office  of  Personnel  Management  recently  issued  amended 
regulations  that  change  the  way  agencies  are  to  evaluate  the  performance 
of  members  of  the  Senior  Executive  Service  (SES).3  Agencies  are  to  place 
increased  emphasis  on  holding  executives  accountable  for  results, 
appraising  executive  performance  on  those  results  balanced  against  other 
perspectives,  and  using  results  as  the  basis  for  performance  awards  and 
other  personnel  decisions.  The  amended  regulations  also  give  agencies 


2  Managing  for  Results:  Experiences  Abroad  Suggest  Insights  for  Federal  Management 
Reforms  (GAO/GGD-95-120,  May  2,  1995). 

3  The  amended  regulations,  65  Fed.  Reg.  60837  -  60845,  can  be  found  in  their  entirety  at 
http://www.opm.gov/fedregis/2000/65-60837-a.pdf. 
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more  flexibility  to  tailor  their  performance  management  systems  to  their 
unique  organizational  requirements  and  climates.  Agencies  must  develop 
systems  to  implement  the  new  policies  for  the  SES  appraisal  cycles  that 
begin  in  2001. 


Scope  and 
Methodology 


As  agreed,  we  examined  VHA  and  DOT  because  of  their  experience  in  using 
performance  agreements  as  part  of  broader  efforts  to  improve  their 
performance.  Our  assessment  of  agencies’  fiscal  year  2000  performance 
plans  produced  under  GPRA  indicated  that  VHA  and  DOT  were  using 
performance  agreements  as  a  mechanism  to  link  employee  performance  to 
organizational  results.4  Additionally,  we  selected  SFA  because  the 
legislation  chartering  it  as  a  PBO  requires  it  to  use  performance 
agreements,  which  we  have  described  as  a  way  to  align  organizational 
goals  and  clarify  accountability.5 

To  meet  our  objective,  we  collected  and  analyzed  these  agencies’  current 
and  previous  performance  agreements;  strategic  plans,  annual 
performance  plans,  and  performance  reports;  personnel  policies  and 
memoranda;  and  other  related  management  documents.  We  discussed  (1) 
VHA’s  performance  agreements  with  the  Chief  Network  Officer  (CNO), 
former  Under  Secretary  for  Health,  and  all  but  1  of  the  22  VISN  directors 
and  spoke  with  the  director’s  designee  at  the  remaining  VISN;  (2)  DOT’s 
performance  agreements  with  the  Deputy  Secretary  of  Transportation,  the 
Assistant  Secretary  for  Administration,  and  knowledgeable  officials  at  each 
of  DOT’s  11  modal  administrations;  and  (3)  SFA’s  performance  agreements 
with  its  COO  and  five  senior  managers.  We  also  contacted  planning  and 
human  capital  officials  at  each  agency  to  discuss  their  experiences  with 
performance  agreements.  We  performed  our  work  in  Washington,  D.C., 
with  additional  visits  to  two  VISN  headquarters  in  the  field,  from  January 
through  September  2000  in  accordance  with  generally  accepted 
government  auditing  standards. 


4  Managing  for  Results:  Opportunities  for  Continued  Improvements  in  Agencies’ 
Performance  Plans  (GAO/GGD/AIMD-99-215,  July  20, 1999) . 

5  Management  Reform:  Elements  of  Successful  Improvement  Initiatives  (GAO/T-GGD-00- 
26,  Oct.  15, 1999). 
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Common  Benefits 
Emerge  From 
Agencies’  Use  of 
Performance 
Agreements 


VHA,  DOT,  and  SFA  used  results-oriented  performance  agreements  with 
their  senior  political  and  career  executives  to  define  accountability  for 
specific  goals,  monitor  progress,  and  contribute  to  performance 
evaluations.  Although  each  agency  developed  and  implemented 
agreements  that  reflected  their  organizational  priorities,  structures,  and 
cultures,  we  identified  five  common  benefits  that  emerged  from  their  use, 


Strengthened  Alignment  of 
Results-Oriented  Goals  With 
Daily  Operations 


High-performing  organizations  know  how  the  products  and  services  they 
deliver  contribute  to  achieving  results.  In  fact,  an  explicit  alignment  of  daily 
activities  with  broader  results  is  one  of  the  defining  features  of  high- 
performing  organizations.  At  the  federal  level,  we  have  found  that  such 
alignment  is  still  very  much  a  work  in  progress.6  Many  agencies  continue  to 
struggle  with  clearly  understanding  how  what  they  do  on  a  day-to-day  basis 
contributes  to  results  outside  of  their  organizations.  The  experiences  of 
VHA,  DOT,  and  SFA  suggest  that  results-oriented  performance  agreements 
can  be  effective  mechanisms  to  define  accountability  for  specific  goals  and 
to  align  daily  activities  with  results. 


For  example,  at  the  Federal  Aviation  Administration  (FAA) ,  a  program 
director’s  goal  to  develop  software  to  help  aircraft  maintain  safe  altitudes 
in  their  approach  paths  is  intended  to  contribute  to  DOT  s  strategic  goal  to 
promote  public  health  and  safety  (see  fig.  2) . 


6  GAO/GGD/AIMD-99-2 1 5,  July  20, 1999. 
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Figure  2:  Translating  DOT  Strategic  Goals  Into  FAA  Programmatic  and  Executive  Goals 


DOT’S 
Strategic  Goals 


Promote  public  health  and  safety  by 
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Develop  the  tracking  and  inspection 
process  for  the  approach  paths  associated 
with  the  Minimum  Safe  Altitude  Warning 
software 


Source:  GAO  analysis  based  on  DOT  and  FAA  planning  documents. 


The  FAA  Administrator’s  performance  agreement  for  fiscal  year  2000 
includes  a  goal  to  reduce  the  commercial  air  carrier  fatal  accident  rate  by 
implementing  the  Safer  Skies  Agenda.  As  part  of  implementing  the  Safer 
Skies  Agenda,  the  Flight  Standards  Service  Director  has  a  goal  to  meet 
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milestones  for  reducing  a  type  of  crash  called  “Controlled  Flight  into 
Terrain,”  which  occurs  when  pilots  lose  their  sense  of  the  plane’s  relation  to 
the  surface  below.  These  milestones  include  validating  Minimum  Safe 
Altitude  Warning  software,  which  is  to  be  developed  by  the  Aviation 
Systems  Standards  Program  Director.  This  software  system  is  expected  to 
aid  air  traffic  controllers  through  both  visual  and  aural  alarms  by  alerting 
them  when  a  tracked  aircraft  is  below,  or  predicted  by  the  computer  to  go 
below,  a  predetermined  minimum  altitude. 

At  VHA,  a  goal  included  in  the  VISN  directors’  performance  agreements  to 
improve  their  VISN’s  scores  in  the  Prevention  Index  (PI)  is  intended  to 
contribute  to  VHA’s,  and  subsequently  the  Department  of  Veterans  Affairs’, 
goal  to  improve  the  overall  health  care  of  veterans.  The  PI  measures  the 
percentage  of  patients  who  receive  eight  medical  interventions,  such  as 
alcohol  screening,  and  is  designed  to  assess  how  well  VHA  follows 
nationally  recognized  approaches  for  primary  prevention  and  early 
detection  of  diseases  with  major  social  consequences,  such  as  alcohol 
abuse.  To  gauge  its  progress  in  this  area,  VHA  has  a  performance  goal  to 
increase  the  scores  on  the  PI,  from  a  fiscal  year  1998  baseline  of  85  percent, 
to  98  percent  of  patients  receiving  the  intervention  by  fiscal  year  2003.  VHA 
has  then  translated  this  goal  into  the  performance  agreements  for  each  of 
VHA’s  VISN  directors.  For  2000,  for  example,  85  percent  of  patients 
receiving  the  interventions  is  considered  “fully  successful”  performance, 
and  90  percent  is  considered  “exceptional”  performance. 

For  SFA,  the  COO’s  goal  to  improve  SFA’s  customer  service  is  intended  to 
contribute  to  the  Department  of  Education’s  strategic  goal  to  ensure  access 
to  postsecondary  education  and  lifelong  learning  and  its  strategic  objective 
that  student  aid  delivery  and  program  management  are  efficient,  financially 
sound,  and  customer-responsive.  This  objective  is  then  translated  into  a 
goal  in  the  COO’s  performance  agreement  to  raise  SFA’s  score  on  the 
American  Customer  Satisfaction  Index.7  SFA’s  score  on  the  index  is  based 
on  surveying  first-time  and  continuing  postsecondary  education  students 
about  the  fairness  of  the  application  process,  usefulness  of  the  information 
that  SFA  provides,  and  professionalism  of  SFA’s  staff.  The  corresponding 
goal  in  the  COO’s  performance  agreement  is  to  raise  SFA’s  score,  from  its 


7  The  American  Customer  Satisfaction  Index  measures  customer  satisfaction  for  public  and 
private  sector  organizations  and  is  produced  by  a  partnership  of  the  University  of  Michigan 
Business  School,  the  American  Society  for  Quality,  and  Arthur  Anderson.  The  highest 
possible  score  on  the  index  is  100. 
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fiscal  year  1999  score  of  63,  to  the  private  sector  industry  average  of  74  by 
2002.  The  COO  then  translates  this  goal  into  the  performance  agreements 
for  each  of  SFA’s  five  senior  managers.  For  example,  in  support  of  this  goal 
to  improve  customer  satisfaction,  the  General  Manager  for  Students  has 
goals  to  increase  aid  awareness  among  students  and  their  families  and  to 
simplify  the  application  process. 


Fostered  Collaboration 
Across  Organizational 
Boundaries 


Collaboration,  interaction,  and  teamwork  across  organizational  boundaries 
are  key  characteristics  of  high-performing  organizations.8  To  this  end,  all 
three  agencies  were  including  results-oriented  goals  in  performance 
agreements  to  encourage  executives  to  work  collaboratively  across 
traditional  organizational  boundaries  or  “silos.”  For  example,  to  encourage 
collaboration,  the  VISN  headquartered  in  Cincinnati  has  implemented 
separate  performance  agreements  that  focus  on  its  patient  services,  such 
as  geriatrics,  in  addition  to  agreements  that  focus  on  traditional 
organizational  boundaries,  such  as  medical  centers.  The  VISN  director  has 
a  performance  agreement  with  each  of  the  VISN’s  “care  line”  directors  for 
patient  services  in  primary  care,  medical  and  surgical  care,  mental  health 
care,  rehabilitation  care,  geriatrics  and  extended  care,  and  clinical  support. 
For  example,  the  mental  health  care  line  director  s  performance  agreement 
includes  mental  health-related  improvement  goals  for  the  entire  VISN.  To 
make  progress  towards  these  mental  health-related  goals,  the  care  line 
director  must  work  across  each  of  the  VISN’s  four  medical  centers  and  with 
corresponding  care  line  managers  at  each  medical  center.  As  part  of  this 
collaboration,  the  care  line  director  needs  to  establish  consensus  among 
VISN  officials  and  external  stakeholders  on  a  strategic  direction  for  the 
mental  health  care  line,  develop  and  implement  integrated  clinical 
programs,  and  allocate  resources  among  the  centers  for  mental  health 
programs. 

DOT  officials  said  that  performance  agreements  provided  a  mechanism  by 
which  to  collaborate  and  interact  on  a  regular  basis  across  modal 
administration  boundaries.  DOT’s  “flagship”  initiatives  designate  certain 
modal  administrations  with  lead  responsibility  for  specific  annual 
performance  goals.  These  lead  administrations  work  with  other 
administrations  to  collaboratively  achieve  the  flagship  goals  by  identifying 
barriers  to  success,  sharing  best  practices,  and  streamlining  procedures. 


8  Human  Capital:  Managing  Human  Capital  in  the  21st  Century  (GAO/T-GGD-OO-77,  Mar. 
9,  2000). 
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For  example,  the  Research  and  Special  Programs  Administration  (RSPA) 
conducts  cross-cutting  research  and  programs  with  other  DOT  modal 
administrations  to  make  the  nation’s  transportation  systems  more 
integrated,  effective,  and  secure.  As  set  forth  in  the  administrator’s 
performance  agreement,  RSPA  has  lead  responsibility — under  DOT’s 
flagship  initiatives — for  DOT’s  safety  goal  to  reduce  the  number  of  serious 
reportable  hazardous  materials  transportation  incidents  and  their  related 
human  and  economic  costs.  To  make  progress  towards  this  goal,  RSPA 
works  with  other  DOT  modal  administrations  that  also  contribute  to  this 
goal.  For  example: 

•  FAA  will  examine  the  feasibility  of  using  new  noninvasive  screening 
technologies  to  detect  hazardous  materials  in  baggage  and  cargo  for  air 
transport. 

•  The  Federal  Highway  Administration,  and  as  of  January  1,  2000,  the 
Federal  Motor  Carrier  Safety  Administration,  will  identify  key  risk 
factors  affecting  highway  safety  for  carriers  transporting  hazardous 
materials. 

•  The  Federal  Railroad  Administration  will  participate  in  developing  and 
conducting  symposiums  on  best  practices  to  address  major  causes  of 
hazardous  materials  incidents,  such  as  non-accident  releases  of 
hazardous  materials  from  rail  tank  cars. 

•  The  Coast  Guard  will  complete  training  of  all  its  units  and  begin  keeping 
records  on  the  occurrence  of  undeclared  shipments  of  hazardous 
materials. 

In  addition  to  these  activities,  these  agencies  formed  a  team  to  evaluate  the 
effectiveness  of  DOT’s  hazardous  materials  safety  program.  The  team 
found  that  the  program  worked  reasonably  well,  but  among  other  things, 
DOT  needed  to  strengthen  its  training  on  industry  safety  practices  and 
compliance  with  regulations  in  order  to  reduce  human  error — the  single 
greatest  contributing  factor  to  hazardous  materials  incidents. 

To  encourage  collaboration  across  its  senior  managers’  operations,  SFA 
officials  told  us  they  plan  to  evaluate  the  senior  managers’  performance 
and  then  make  bonus  decisions  based  on  (1)  SFA’s  overall  performance  and 
(2)  the  extent  to  which  the  senior  manager  meets  the  goals  in  his  or  her 
performance  agreement.  For  example,  for  the  General  Manager  for 
Students,  50  percent  of  his  or  her  performance  evaluation  will  be  based  on 
his  or  her  contribution  towards  SFA’s  overall  performance  goals  for 
customer  satisfaction,  unit  cost,  and  employee  satisfaction.  The  remaining 
50  percent  will  be  based  on  progress  towards  the  goals  in  his  or  her 
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agreement  that  specifically  relate  to  students,  which  are  to  improve 
customer  satisfaction  among  aid  applicants  and  their  families,  reduce  his 
or  her  organization’s  unit  costs,  and  improve  satisfaction  among  employees 
who  work  to  support  student  services. 


Enhanced  Opportunities  to 
Discuss  and  Routinely  Use 
Performance  Information  to 
Make  Program 
Improvements 


High-performing  organizations  use  performance  information  on  a  routine 
basis  to  pinpoint  opportunities  to  improve  their  performance.  Towards  this 
end,  the  experiences  of  VHA,  DOT,  and  SFA  show  how  results-oriented 
performance  agreements  can  be  an  effective  vehicle  to  foster  routine 
communication  about  progress  towards  achieving  goals,  existing  and 
emerging  performance  gaps,  and  strategies  to  better  achieve  results. 
Officials  at  all  three  agencies  noted  that  data  on  progress  in  meeting  goals 
in  the  agreements  were  examined  in  light  of  the  internal  or  external  factors 
beyond  the  control  of  the  agency  that  can  influence  the  degree  to  which 
goals  are  achieved.  This  contributed  to  more  specific  discussions  and 
decisions  to  mitigate  or  leverage  these  internal  or  external  factors,  as 
appropriate,  to  better  achieve  results.  In  addition,  this  more  routine  use  of 
performance  information  highlighted  the  implementation  issue  of  making 
performance  information  more  accessible  and  helpful  to  executives.  For 
example,  VHA’s  CNO  meets  quarterly  with  each  VISN  director  to  discuss 
that  VISN’s  progress  towards  the  goals  in  the  director’s  performance 
agreement.  At  these  meetings,  the  CNO  and  directors  also  discuss  the 
impact  of  internal  or  external  factors  and  strategies  to  address  their  effects. 
To  inform  these  meetings,  VHA  produces  a  quarterly  Network  Performance 
Report  that  presents  both  VHA-wide  and  VISN-specific  progress  on  each  of 
the  performance  goals  in  the  VISN  directors’  performance  agreements.  In 
turn,  some  VISNs  also  report  information  on  a  more  frequent  basis  to 
communicate  progress,  identify  performance  gaps,  and  consider  possible 
actions  to  improve  performance. 


VHA’s  quarterly  Network  Performance  Report,  an  example  of  which  is 
shown  in  figure  3,  shows  each  of  the  22  VISNs’  progress  towards  the  goals 
in  the  VISN  directors’  agreements,  such  as  the  goal  for  follow-up  care  after 
hospitalization  for  mental  illness.  For  example,  90  percent  of  the  patients 
within  VISN  5,  which  is  headquartered  in  Baltimore,  received  follow-up 
care  for  the  third  quarter  of  fiscal  year  2000.  In  between  VHA’s  quarterly 
reports,  the  Baltimore  VISN  also  produces  biweekly  reports  that  allow  it  to 
monitor  even  more  frequently  progress  on  a  consistent  set  of  goals  by  its 
three  medical  centers.  As  shown  in  the  lower  part  of  figure  3,  the  VISN’s 
biweekly  performance  report  for  August  2000  allowed  it  to  see  that  its 
VISN-wide  rate  remained  at  90  percent;  its  medical  center  at  Martinsburg, 
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WV  was  at  91  percent;  its  Veterans  Affairs  Maryland  Health  Care  System 
(VAMHCS)  was  at  89  percent;  and  its  medical  center  at  Washington  was  at 
91  percent.  For  this  goal,  a  VISN’s  performance  is  fully  successful  when  75 
percent  of  discharged  patients  received  follow-up  care  after  hospitalization 
for  mental  illness,  and  its  performance  is  exceptional  when  90  percent  of 
patients  received  follow-up  care. 
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Figure  3:  Excerpt  of  Performance  Information  on  Follow-up  Care  After  Hospitalization  for  Mental  Illness  in  VHA's  Quarterly 
Network  Performance  Report  and  VISN  5’s  Biweekly  Performance  Report 
VHA  Quarterly  Network  Report 


a“FS”  stands  for  fully  successful,  which  a  VISN  attains  when  75  percent  of  its  discharged  patients 
receive  follow-up  care. 

b“E”  stands  for  exceptional,  which  is  attained  when  90  percent  of  discharged  patients  receive  follow-up 
care. 

Source:  VHA  Quarterly  Network  Performance  Report,  May  8,  2000;  and  VISN  5’s  Biweekly 
Performance  Report,  August  17,  2000. 
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The  Baltimore  VISN’s  Executive  Leadership  Council,  which  comprises  the 
VISN  director,  its  medical  center  directors,  and  other  staff,  such  as  nurse 
executives,  uses  the  VISN’s  biweekly  performance  report  during  its 
monthly  meetings  to  discuss  the  VISN  s  progress.  VISN  officials  said  that 
having  this  performance  information  routinely  available  provided  a 
foundation  to  discuss  the  VISN’s  progress  each  month  and,  where 
appropriate,  discuss  strategies  to  address  any  performance  gaps. 

As  agencies  use  performance  agreements  to  enhance  opportunities  to 
communicate  progress  and  discuss  strategies  to  address  performance  gaps, 
they  have  been  pushed  to  make  performance  information  more  accessible 
to  executives  in  a  time  frame  and  format  that  corresponds  to  what  is  most 
helpful  to  them.  For  example,  in  attempts  to  provide  timely  and  readily 
accessible  performance  information,  VHA  provides  VISNs  access  to  the 
VISN  Support  Service  Center,  which  has  a  Web  site  that  offers  on-line 
access  to  reports  and  databases  related  to  a  range  of  VISN  operations, 
including  VISN  progress  towards  performance  goals.  According  to  VHA, 
this  allows  VISNs  to  access  and  use  performance  information  on  a  more 
frequent  and  timely  basis  than  was  possible  with  previous  VHA  data 
collection  and  reporting  systems.  To  further  encourage  staff  to  access 
performance  information  more  frequently,  some  VISNs  have  also 
developed  their  own  intranet-based  data  systems  to  make  data  more  easily 
available  to  staff. 

VHA  can  continue  to  make  progress  in  providing  timely  and  readily 
accessible  performance  information.  For  example,  some  VISN  directors 
told  us  that  they  still  need  to  receive  more  timely  performance  information 
on  additional  measures,  such  as  customer  satisfaction.  Specifically,  VHA 
collects  data  for  customer  satisfaction  annually,  and  officials  indicated  that 
receiving  data  so  infrequently  made  it  difficult  to  change  strategies  during 
the  year  to  improve  their  VISNs’  customer  satisfaction  ratings.  In  addition, 
as  illustrated  by  our  recent  report  on  VHA’s  patient  waiting  times,  it  needs 
to  continue  its  efforts  to  develop  reliable  performance  information  that  is 
readily  accessible  to  all  of  its  VISNs.9 

Performance  agreements  at  DOT — and  the  bimonthly  meetings  between 
the  Deputy  Secretary  and  modal  administrators  to  discuss  progress 
towards  the  goals  and  activities  in  the  agreements — also  provided 


9  Veterans  ’  Health  Care:  VA  Needs  Better  Data  on  Extent  and  Causes  of  Waiting  Times 
(GAO/HEHS-00-90,  May  31,  2000). 
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opportunities  to  enhance  communication  about  results,  identify 
performance  gaps,  and  then  consider  actions  the  modal  administrations 
can  take  to  further  improve  performance.  For  example,  the  1999 
performance  agreement  for  the  National  Highway  and  Traffic  Safety 
Administration’s  (NHTSA)  Administrator  included  a  goal  to  “continue 
progress  toward  meeting  or  exceeding  the  President  s  goal  to  increase 
safety  belt  use  nationwide  by  85  percent  by  2000  and  90  percent  by  2005.” 
Nationwide  seatbelt  usage,  however,  hovered  between  68  and  70  percent 
from  1995  to  1998  and  declined  slightly  to  67  percent  in  1999.  Although 
DOT  remains  committed  to  the  national  seat  belt  goal,  NHTSA  officials  told 
us  they  will  need  to  rethink  strategies  to  meet  the  goal  and  focus  on 
increasing  seat  belt  use  among  groups  with  lower  use  rates  and  expanding 
information  and  educational  campaigns  to  target  diverse  audiences.  The 
strategies  NHTSA  plans  to  use  in  the  coming  year  to  increase  seat  belt  use 
include  expanding  its  Buckle  Up  America  Campaign  to  all  50  states  and 
working  with  states  to  strengthen  their  programs  and  pass  primary 
enforcement  of  seat  belt  laws.  Officials  said  they  will  then  evaluate  the 
success  of  these  initiatives  to  determine  if  further  adjustments  are  needed 
to  achieve  the  2001  seat  belt  use  goal. 

In  order  to  further  enhance  communication  about  progress  towards  its 
goals,  DOT  is  seeking  to  implement  a  revised  and  streamlined  system  that 
will  be  more  widely  accessible  within  each  modal  administration. 
Currently,  DOT’s  Performance  Agreement  Tracking  System  tracks  and 
provides  information  for  each  of  its  executives’  performance  agreements. 
DOT  officials  said  this  current  system  has  evolved  into  multiple  databases 
that  do  not  efficiently  sort  and  analyze  results  across  DOT,  which  detracts 
from  the  usefulness  of  the  performance  information. 


Provided  Results-Oriented 
Basis  for  Individual 
Accountability 


High-performing  organizations  hold  executives  accountable  for 
contributing  to  the  achievement  of  results-oriented  goals.10  In  pursuit  of 
this  characteristic,  officials  at  VHA  and  SFA  told  us  that  performance 
agreements  were  a  vehicle  to  bring  results-oriented  performance 
information  into  evaluations  of  executive  performance.  For  example,  VHA 
officials  said  that  progress  towards  the  goals  in  the  agreements  played  an 
important  role  in  determining  the  VISN  director  performance  evaluations 
and  bonuses — which  in  1999  were  up  to  $20,000.  Progress  towards  VHA’s 


10 Human  Capital:  Key  Principles  From  Nine  Private  Sector  Organizations  (GAO/GGD-00- 
28,  Jan.  31,  2000). 
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health  care-related  goals  accounts  for  50  percent  of  the  agreement,  areas  of 
special  interest  30  percent,  and  the  core  managerial  competencies  20 
percent.  VISN  directors  also  submit  self-assessments  of  their  performance 
that  include  examples  of  actions  or  accomplishments,  which  VHA  also 
considers  in  director  evaluations. 

Similarly,  SFA  said  performance  agreements  play  a  major  role  in  its 
evaluation  process.  Progress  towards  the  goals  in  performance  agreements 
is  the  basis  by  which  SFA  determines  how  much  of  the  allowed  bonuses  its 
COO  and  senior  managers  receive.  According  to  SFA’s  PBO  legislation,  the 
COO  is  entitled  to  a  bonus  of  about  $61,000  in  2000,  and  the  senior 
managers  are  entitled  to  salaries  and  bonuses  of  up  to  $153,000  in  2000. 
Specifically,  the  COO  s  evaluation  is  based  on  SFAs  progress  towards  the 
three  goals  in  the  COO  s  performance  agreement.  According  to  SFA 
officials,  SFA  plans  to  evaluate  its  senior  managers’  performance  on  the 
extent  to  which  the  senior  manager  meets  the  goals  in  his  or  her 
performance  agreement,  as  well  as  SFA’s  overall  performance  in  customer 
satisfaction,  unit  cost,  and  employee  satisfaction.  Under  its  PBO 
legislation,  SFA  is  required  to  submit  copies  of  both  the  COO  and  senior 
manager  evaluations  to  Congress  along  with  its  annual  performance  report. 

Whereas  VHA  and  SFA  consider  progress  towards  the  goals  in  performance 
agreements  as  part  of  the  process  to  determine  executive  ratings  and 
bonuses,  DOT’s  modal  administrators,  however,  are  appointed  by  the 
President  and  confirmed  by  the  Senate,  so  they  are  not  eligible  for  cash 
awards.  Since  1998,  however,  the  Deputy  Secretary  has  “tallied”  for  each 
administration  the  percentage  of  activities  it  accomplished  that  are 
intended  to  achieve  DOT’s  goals.  The  tally  also  showed  where  each 
administrator  stood  relative  to  other  administrators  in  achieving  the  goals 
in  his  or  her  agreement,  which  in  turn  underscored  to  the  administrators 
the  importance  of  their  performance  agreements. 


Maintained  Continuity  of 
Program  Goals  During 
Leadership  Transitions 


High-performing  organizations  recognize  that  they  need  to  reinforce 
accountability  for  organizational  goals  during  times  of  leadership 
transitions.  To  this  end,  DOT  and  VHA  noted  that  their  results-oriented 
performance  agreements  helped  provide  continuity  during  times  of 
leadership  transition  by  maintaining  a  consistent  focus  on  a  set  of  broad 
programmatic  priorities.  They  said  that  because  the  agreements  clearly  and 
concisely  outlined  top  leadership  priorities  during  a  given  year,  the 
agreements  served  as  a  convenient  vehicle  for  new  leadership  to  identify 
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and  maintain  focus  on  the  most  pressing  issues  confronting  the 
organization. 

More  generally,  the  existence  of  an  established  process  for  developing  and 
using  performance  agreements  provides  new  leadership  with  a  tested  tool 
that  it  can  use  to  communicate  its  priorities  and  instill  those  priorities 
throughout  the  organization.  VHA,  DOT,  and  SFA  are  in  the  initial  phases  of 
implementing  performance  agreements  beyond  the  senior  political  and 
career  executive  levels  and  are  exploring  how  far  and  how  best  to  cascade 
them. 

•  For  staff  at  lower  levels  of  VISNs,  agreements  are  being  cascaded  to 
varying  extents — for  example  to  facility  directors,  chiefs  of  staff,  and 
service-line  directors. 

•  For  fiscal  year  2000,  DOT  is  requiring  all  modal  administrations  and 
other  offices,  as  appropriate,  to  align  the  performance  plans  of  the 
associate  administrators,  executive  directors,  and  other  SES  who  report 
directly  to  modal  administrators  with  their  respective  administrator’s 
performance  agreement. 

•  SFA’s  PBO  legislation  also  requires  it  to  establish  a  performance 
management  system  that  creates  goals  for  employee,  group,  or 
organizational  performance  that  are  consistent  with  the  PBO  s 
performance  plan.  According  to  SFA  officials,  SFA  is  in  the  initial  stages 
of  developing  this  system. 

Looking  at  potential  SES  retirements  underscores  the  importance  of  having 
stabilizing  mechanisms,  such  as  performance  agreements,  in  place  to 
ensure  continuity  through  leadership  transitions.  We  recently  found  that 
the  federal  government  as  a  whole  may  need  to  replace  a  substantial 
number  of  career  SES  members  who  will  become  eligible  to  retire  between 
September  30, 1999,  and  September  30,  2005. 11  Although  not  all  of  this 
group  of  SES  members  who  become  eligible  will  actually  retire  within  that 
time,  a  sizeable  percentage  likely  will.  The  Office  of  Personnel  Management 
estimates  that  of  the  almost  6,000  career  SES  members,  about  45  percent 
will  retire  during  fiscal  years  1999  through  2005. 


11  Senior  Executive  Service:  Retirement  Trends  Underscore  the  Importance  of  Succession 
Planning  (GAO/GGD-00-1 13BR,  May  12,  2000). 
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Concluding 

Observations 


As  agencies  continue  to  shift  towards  a  greater  focus  on  results,  there  is 
also  growing  recognition  on  the  part  of  Congress  and  the  executive  branch 
that  agencies  need  to  make  progress  connecting  employee  performance 
with  agency  success.  The  emerging  benefits  we  identified  from  VHAs, 
DOT’S,  and  SFA’s  experiences  suggest  that  performance  agreements  are  one 
management  tool  agencies  can  use  as  they  seek  to  improve  performance 
and  accountability.  The  experiences  of  VHA,  DOT,  and  SFA  also  suggest 
that  as  agencies  move  forward  in  using  performance  agreements,  they  will 
need  to  address  several  key  implementation  issues,  as  highlighted 
throughout  this  report. 

For  example,  our  work  has  shown  that  many  agencies  continue  to  struggle 
with  clearly  understanding  how  what  they  do  on  a  day-to-day  basis 
contributes  to  results  outside  of  their  organization.  VHA,  DOT,  and  SFA 
used  performance  agreements  to  strengthen  the  alignment  of  daily 
operations  with  results-oriented  performance  goals.  In  addition,  while  the 
experiences  of  VHA,  DOT,  and  SFA  show  how  performance  agreements  can 
be  effective  vehicles  to  foster  routine  communication  about  progress 
towards  goals,  existing  and  emerging  performance  gaps,  and  strategies  to 
better  achieve  results,  their  experiences  also  underscore  the  need  to 
ensure  that  executives  and  managers  have  the  performance  information 
they  need,  when  they  need  it,  and  in  a  format  that  meets  their  needs.  As 
such,  agencies  are  seeking  to  improve  the  timeliness  and  accessibility  of 
data  to  executives  and  line  managers.  Finally,  all  three  agencies  appreciate 
that  to  be  fully  effective,  performance  agreements  need  to  cascade  beyond 
the  senior  political  and  executive  level  into  lower  organizational  levels.  All 
three  are  therefore  undertaking  initial  efforts  in  this  regard.  As  part  of  these 
initial  efforts,  they  are  exploring  how  far  and  how  best  to  cascade  the  use 
of  agreements. 

Nevertheless,  the  experiences  of  VHA,  DOT,  and  SFA  suggest  that  as  they 
and  other  agencies  continue  to  gain  experience  with  performance 
agreements,  it  appears  that  such  agreements  can  become  an  increasingly 
vital  part  of  overall  efforts  to  improve  programmatic  performance  and 
better  achieve  results. 


A  /^\7  F  OTTjTYiQtltS  We  provided  drafts  of  this  report  in  October  2000  to  the  Acting  Secretary  of 

^  v  Veterans  Affairs,  the  Secretary  of  Transportation,  SFA’s  Chief  Operating 

Officer,  and  the  Secretary  of  Education,  or  their  designees,  for  their  review 
and  oral  comment.  They  responded  through  their  GAO  liasons  that  they 
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agreed  with  the  contents  of  the  draft  report  and  in  some  cases  provided 
clarifying  technical  comments.  Where  appropriate,  we  have  made  changes 
to  this  report  that  reflect  these  technical  comments. 


As  agreed  with  your  office,  unless  you  publicly  announce  its  contents 
earlier,  we  plan  no  further  distribution  of  this  report  until  30  days  from  the 
date  of  this  letter.  At  that  time,  we  will  provide  copies  of  this  report  to 
Representative  Henry  Waxman,  Senator  Fred  Thompson,  and  Senator 
Joseph  Lieberman  in  their  respective  capacities  as  the  Ranking  Minority 
Member  of  the  Committee  on  Government  Reform,  Chairman  of  the  Senate 
Committee  on  Governmental  Affairs,  and  Ranking  Minority  Member  of  the 
Senate  Committee  on  Governmental  Affairs;  the  heads  of  the  agencies  we 
reviewed;  the  Honorable  Janice  Lachance,  Director  of  the  Office  of 
Personnel  Management;  and  other  interested  parties.  We  will  also  make 
this  report  available  to  others  upon  request. 

If  you  have  any  questions  about  this  report,  please  contact  me  or  Lisa 
Shames  on  (202)  512-6806.  Justin  Brown  and  Janice  Lichty  were  key 
contributors  to  this  report. 


Sincerely  yours. 


J.  Christopher  Mihm 
Director,  Strategic  Issues 
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VHA’s  Implementation  of 
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The  Veterans  Health  Administration  (VHA)  operates  an  approximately  $18 
billion  health  care  system  and  serves  the  medical  care  needs  of  veterans  by 
providing  primary  and  specialized  care  at  hundreds  of  service  delivery 
locations  across  the  country.  Between  1946  when  VHA  was  established, 
and  1995,  VHA  grew  into  the  nations  largest  direct  provider  of  health  care. 
VHA  served  veterans  at  more  than  600  locations  and  focused  primarily  on 
hospital  care,  the  use  of  high  technology,  and  medical  specialization. 
During  this  time,  however,  our  work  showed  that  VHA  did  not  keep  pace 
with  societal  and  industry  changes,  such  as  a  market-based  restructuring  of 
American  health  care  and  the  increase  of  managed  care.1 

Beginning  in  October  1995,  VHA  sought  to  shift  its  emphasis  from  being  a 
hospital  operator  to  a  health  care  provider  that  relied  on  community-based, 
integrated  networks.  In  fiscal  year  1996,  VHA  furthered  this  transformation 
by  decentralizing  its  management  structure  from  4  regions  to  22  Veterans 
Integrated  Service  Networks  (VTSN).  The  VISNs  were  designed  to 
coordinate  the  activities  of  hundreds  of  hospitals,  outpatient  clinics, 
nursing  homes,  and  other  facilities  within  a  given  geographic  area.  VHA 
expected  the  VISN  structure  to  improve  efficiency  by  reducing  duplicative 
services  and  shifting  services  from  costly  inpatient  care  to  less  costly 
outpatient  care.  To  accomplish  its  reform  goals,  VHA  gave  each  VISN 
substantial  operational  autonomy  and  established  performance  goals  to 
hold  network  and  medical  center  directors  accountable  for  achieving 
performance  improvements. 


Define  Accountability  for  One  key  VHA  initiative  to  drive  its  reforms  was  to  have  each  of  the  22  VISN 
Specific  Goals  directors  enter  into  a  performance  agreement  with  the  Under  Secretary  for 

”  Health.  VHA  began  implementing  these  performance  agreements  in  1996. 

These  performance  agreements  included  performance  goals  with  specific 
targets  that  each  VISN  was  responsible  for  accomplishing  during  the  next 
year.  The  VISN  director;  the  Chief  Network  Officer  (CNO),  whose  office 
administers  the  performance  agreements;  and  the  Under  Secretary  for 
Health  initiate  the  agreement  at  the  beginning  of  each  year.  As  illustrated  in 
figure  4,  the  agreements  for  fiscal  year  2000  consist  of  three  weighted 
parts — A,  B,  and  C. 


1  Veterans  Affairs:  Progress  and  Challenges  in  Transforming  Health  Care  (GAO/T-HEHS- 
99-109,  April  15,  1999). 
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•  Part  A  accounts  for  20  percent  of  the  agreement  and  outlines  “core 
competencies”  that  define  the  behaviors  VISN  directors  are  expected  to 
exhibit  as  they  manage  the  VISN  during  the  coming  year.  Core 
competencies  within  Part  A  include  such  areas  as  interpersonal 
effectiveness,  flexibility/adaptability,  and  technical  competency. 

•  Part  B  accounts  for  30  percent  of  the  agreement  and  outlines  “areas  of 
special  interest”  that  reflect  priority  areas  for  VHA  during  the  coming 
year.  Areas  of  special  emphasis  within  Part  B  include  such  areas  as 
patient  safety. 

•  Part  C  accounts  for  50  percent  of  the  agreement  and  contains  the 
majority  of  the  performance  goals  that  gauge  each  VISN’s  progress 
towards  VHA’s  health  care  mission.  Part  C  is  further  divided  into  VHA’s 
five  domains  for  health  care:  quality,  functional  status,  satisfaction, 
access,  and  cost.  Within  each  of  these  domains,  the  agreement  sets 
measurable  goals  for  each  VISN.  For  most  goals,  the  agreement  also  sets 
achievement  levels  that  VISNs  are  expected  to  attain  for  “fully 
successful”  and  “exceptional”  performance.  For  example,  under  the 
domain  of  access,  VISNs  were  to  decrease  the  average  waiting  time  for 
patients  to  obtain  appointments  in  ophthalmology,  audiology, 
orthopedics,  cardiology,  urology,  and  primary  care  by  September  30, 
2000.  For  this  goal,  a  VISN’s  performance  was  fully  successful  at  45  days 
and  exceptional  at  30  days  or  less. 
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Figure  4:  Excerpts  of  VHA  Performance  Agreement  Template  for  2000 


Part  A:  Core  Competencies 

I.  Interpersonal  Effectiveness:  The  ability  to  build  and  sustain  relationships,  resolve  conflict,  handle 
negotiation  effectively,  and  develop  collaborative  working  relationships.  The  successful  executive 
displays  empathy,  empowers  others,  and  possesses  written  and  oral  communication  skills. 

IV.  Flexibility/Adaptability:  The  ability  to  quickly  adapt  to  change,  handle  multiple  inputs  and  tasks 
simultaneously,  and  accommodate  new  situations  and  realities.  The  successful  executive  works 
well  with  all  levels  and  types  of  people,  welcomes  divergent  ideas,  and  maximizes  limited  resources. 

VIII.  Technical  Competency:  The  knowledge  and  skills  to  perform  and  evaluate  the  work  of  the  organization 
based  upon  a  clear  understanding  of  the  processes,  procedures,  standards,  methods,  and  technologies 
of  the  organization.  The  successful  executive  demonstrates  functional  and  technical  literacy  and 
measures  results  of  work. 


Part  B:  Areas  of  Special  Interest 

4.  Patient  Safety: 

a.  All  facilities  will  participate  in  distribution  of  the  National  Center  for  Patient  Safety’s  culture  and 
attitudina!  survey  addressing  patient  safety. 

b.  Each  facility  will  have  at  least  one  person  trained  in  performing  Root  Cause  Analysis  (RCA)  in 
accordance  with  process  as  defined  by  the  National  Center  for  Patient  Safety. 
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Part  C:  Performance  Measures 


Healthcare  Value:  Domain  of  Quality 
1.  Prevention  Care  Index  (PI) 

By  September  30, 2000,  the  performance  on  the  PI  will  increase. 

Achievement  levels 

Fully  Successful:  PI  is  85%  based  on  entire  FY  2000  data 
Exceptional:  PI  is  90%  based  on  entire  FY  2000  data 


Healthcare  Value:  Domain  of  Access 
8.  Waiting  Times 

By  September  30,  2000,  the  average  waiting  time  will  decrease  for  the  following  DSS  identifier 
categories  (clinics):  ophthalmology,  audiology,  orthopedics,  cardiology,  urology,  and  Primary  Care. 

Achievement  levels 

Fully  successful:  45  days 
Exceptional:  30  days  or  less 


Network  Director  Date 


Chief  Network  Officer  Date 


Source:  VHA  2000  Network  Performance  Plan. 
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In  addition  to  the  goals  in  the  agreements,  the  CNO  uses  a  list  of  “monitors” 
that  represent  important  areas  of  focus  for  VISNs.  These  monitors  can 
cover  areas  that  for  the  most  part  have  been  accomplished,  yet  still  deserve 
attention;  or  they  can  also  be  used  to  focus  on  certain  areas  that  may  be 
emerging  departmental  priorities.  Recent  monitors  have  included  the 
percentage  of  long-term  care  patients  who  received  pneumococcal 
vaccinations  and  the  percentage  of  patients  in  ambulatory  or  long-term 
care  who  have  received  a  pain  assessment.  The  monitors  can  also  be 
tailored  to  local  needs  and  VISNs  can  use  monitors  to  highlight  up  to  six 
initiatives  that  they  believe  require  attention. 

Each  year,  the  CNO  and  the  Office  of  Performance  and  Quality  refine  the 
previous  year’s  performance  agreement  template.  These  two  offices,  along 
with  the  Performance  Measurement  Working  Group,  which  comprises 
VISN  directors  and  other  staff,  work  to  develop  the  goals  and  measures  for 
the  VISN  director  performance  agreements.  To  define  the  performance 
measures  that  are  included  in  the  VISN  director  agreements,  VHA  provides 
a  document  that  provides  definitions  and  data  collection  strategies  for  each 
goal  in  the  agreement,  with  a  particular  focus  on  health  care-related 
performance  goals. 


Monitor  Progress  During  the  VHA  has  a  standardized  approach  to  monitoring  VISN  directors’  progress 
year  during  the  year  that  consists  of  formal  reporting  of  progress  towards  the 

health  care-related  goals  and  areas  of  special  interest  included  in  their 
agreements,  which  is  then  followed  by  meetings  to  discuss  performance. 
The  Office  of  Performance  and  Quality  produces  a  network  performance 
report  each  quarter,  which  is  provided  to  each  VISN.  This  report  provides  a 
status  of  each  VISNs  and  VHA’s  overall  progress  towards  the  goals  included 
in  the  performance  agreements.  Following  these  quarterly  reports,  the  CNO 
meets  with  each  VISN  director  to  discuss  progress  towards  the  goals  and 
any  internal  or  external  factors  that  may  be  affecting  progress.  VHA 
officials  told  us  that  these  meetings  are  valuable  opportunities  to  monitor 
progress  and  discuss  performance. 

To  support  their  efforts  to  monitor  progress,  VHA  has  several  data  systems. 
VISNs  have  access  to  the  VISN  Support  Service  Center,  which  has  a  Web 
site  that  allows  both  VHA  headquarters  and  VISN  officials  access  to  reports 
and  databases  on  these  systems  that  track  VISN  progress  towards  some  of 
the  goals  in  the  performance  agreements.  VHA  also  uses  these  collection 
systems  to  produce  the  quarterly  performance  reports.  To  verify  and 
validate  some  of  its  performance  information,  VHA  has  an  External  Peer 
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Review  Program,  which  is  designed  to  provide  a  national  database  that 
ensures  more  objective  performance  information.  As  part  of  this  program, 
an  independent  contractor  assesses  the  data’s  accuracy  by  reviewing 
samples  of  patient  records  and  then  generates  data  flies  and  reports  for 
each  quarter  of  the  fiscal  year  on  specific  aspects  of  clinical  care. 


To  evaluate  the  performance  of  VISN  directors  at  the  end  of  each  fiscal 
year,  the  CNO,  Under  Secretary  for  Health,  and  Performance  Review  Board 
use  both  quantitative  and  qualitative  performance  information  to 
contribute  to  their  judgement  when  making  evaluation  decisions.  The 
quantitative  performance  information  includes  progress  towards  the  health 
care-related  goals  and  areas  of  special  interests  included  in  the 
performance  agreements.  The  qualitative  performance  information 
includes  the  VISN  director’s  self-assessment  of  his  or  her  performance 
during  the  year  that  describes  actions  or  accomplishments  that  reflect  each 
competency  and  the  CNO’s  assessment  of  the  director’s  management 
competencies  included  in  the  agreement.  VHA  officials  indicated  that 
having  the  quantitative  performance  information  related  to  the  agreements 
helped  them  assess  VISN  director  performance  when  making  performance 
rating  and  bonus  decisions.  In  1999,  VISN  directors  could  receive  bonuses 
of  up  to  $20,000. 

A  key  aspect  of  VHA’s  evaluation  of  VISN  directors  is  its  annual  network 
performance  agreement  report,  which  takes  the  place  of  a  fourth-quarter 
report.  Most  recently,  the  1999  performance  report  included  a  summary  of 
VHA’s  overall  progress  towards  the  goals  in  the  VISN  director  performance 
agreements  and  individual  VISN  progress  towards  the  goals.  Charts  and 
graphs  allowed  senior  political  and  career  executives,  as  well  as 
employees,  to  compare  each  VISN  to  the  entire  agency  and  to  other  VISNs 
across  the  country. 

A  VHA  official  indicated  that  although  VHA  has  not  evaluated  the  discrete 
contributions  performance  agreements  have  made  to  its  performance, 
including  corresponding  goals  in  the  performance  agreements  of  VISN 
directors  contributed  to  improvements  in  those  key  organizational  goals. 
For  example,  between  fiscal  years  1997  and  1999,  VHA  reported  that  its 
performance  on  the  Prevention  Index  had  improved  from  69  to  81  percent. 
A  goal  requiring  VISNs  to  produce  measurable  increases  in  the  Prevention 
Index  has  been  included  in  the  directors’  performance  agreements  each 
year  between  1997  and  1999. 


Contribute  to  Performance 
Evaluations 
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To  further  define  accountability  at  lower  levels,  most  VISNs  are 
implementing  performance  agreements  between  the  VISN  directors  and 
individuals  reporting  to  them,  such  as  medical  center  directors,  clinical 
managers,  and  quality  managers.  Some  medical  center  directors  also  then 
have  agreements  with  individuals  who  report  to  them,  such  as  associate 
directors  or  chiefs  of  staff. 
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The  Department  of  Transportation  (DOT)  has  for  a  number  of  years  faced 
critical  challenges  in  its  efforts  to  help  ensure  the  safe  and  efficient 
movement  of  people  and  the  cost-effective  investment  of  resources  in  the 
nation’s  transportation  infrastructure.  Although  DOT  has  contributed  to 
improving  the  nation’s  transportation  systems,  we  have  reported  that  it  has 
also  experienced  problems,  such  as  cost  overruns  and  delays  in  air  traffic 
control  modernization  programs  and  highways  and  transit  projects.1 

According  to  DOT,  its  efforts  to  implement  performance  agreements  were 
in  response  to  the  National  Performance  Review’s  initiative  to  improve 
performance  and  accountability  at  executive  agencies  and  a  specific  1993 
recommendation  to  “develop  performance  agreements  with  senior  political 
leadership  that  reflect  organizational  and  policy  goals.”  In  1994,  all  DOT 
administrators  were  asked  to  sign  DOT’s  first  performance  agreement, 
which  was  between  the  Secretary  of  Transportation  and  the  President. 
DOT  has  implemented  performance  agreements  with  its  administrators  in 
each  subsequent  year. 


Define  Accountability  for  As  of  1999,  DOT  had  implemented  a  total  of  26  annual  performance 
SoeciflC  Goals  agreements  with  its  modal  administrators,  all  Assistant  Secretaries,  and 

**  Office  Directors  within  the  Office  of  the  Secretary.  DOT  has  a  template  for 

its  performance  agreements  that  generally  dovetails  its  annual 
performance  plans  produced  under  the  Government  Performance  and 
Results  Act  (GPRA).  According  to  DOT  officials,  this  helps  ensure  that  the 
performance  agreements  are  consistent  with  DOT’s  goals.  The  agreements 
include  organization-specific  activities  that  are  collectively  intended  to 
achieve  DOT’s  performance  goals,  as  well  as  “corporate  management 
strategies”  that  cut  across  organizational  boundaries  and  set  additional 
expectations  for  administrators,  such  as  to  use  customer  feedback  to 
improve  programs. 


Monitor  Progress  During  the  DOT’S  approach  to  monitoring  executives’  progress  during  the  year 
Year  consists  of  formal  reporting  of  progress  towards  the  activities  in  the 

administrator’s  agreements,  which  is  then  followed  by  meetings  to  discuss 
performance.  Each  modal  administration  produces  a  performance  report 
to  prepare  for  regular  meetings  between  the  Deputy  Secretary  and  the 


1  Major  Management  Challenges  and  Program  Risks:  Department  of  Transportation 
(GAO/OCG-99-13,  Jan.  1999). 
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modal  administrator.  These  meetings  occur  on  at  least  a  bimonthly  basis. 
At  these  meetings,  the  Deputy  Secretary  uses  the  performance  reports  as 
the  basis  to  discuss  progress  against  the  goals  in  the  performance 
agreements.  The  Deputy  Secretary  and  each  administrator  also  discuss 
internal  or  external  factors  that  may  be  affecting  progress  and  actions  that 
can  be  taken  to  improve  performance. 

DOT’s  Performance  Agreement  Tracking  System  (PATS)  is  used  to  produce 
each  administration’s  monthly  performance  report.  The  PATS  includes  a 
database  for  each  modal  administrator  and  the  other  Assistant  Secretaries 
and  Directors  who  have  an  agreement  with  the  Secretary.  DOT  officials 
indicated  DOT  is  seeking  to  make  PATS  more  widely  accessible  within  each 
modal  administration. 

Other  groups  within  DOT  also  monitor  progress  towards  the  goals  in  the 
agreements.  For  example,  the  Senior  Leadership  Team,  which  comprises 
modal  administrators  and  assistant  secretaries,  meets  each  week  to 
discuss,  among  other  issues,  progress  towards  activities  in  the 
administrator’s  agreements.  The  modal  administrations  also  reported  using 
the  PATS  reports  for  a  number  of  meetings  and  discussions  of  their 
performance. 


Contribute  to  Performance  Since  1998  the  Deputy  Secretary  has  “tallied”  for  each  administration  the 
Evaluations  percentage  of  activities  listed  in  the  administrator’s  performance 

agreement  it  accomplished.  The  tally  also  included  completion  rates  for 
DOT’s  strategic  goals  and  corporate  management  strategies.  DOT’s  modal 
administrators  are  appointed  by  the  President  and  confirmed  by  the 
Senate,  thus  they  are  not  eligible  for  cash  awards. 

As  part  of  an  overall  effort  to  evaluate  and  improve  its  accountability 
systems  and  its  progress  towards  achieving  strategic  goals,  DOT  found  in 
1999  that  its  system  of  performance  planning  and  performance  agreements 
“do  an  excellent  job  of  communicating  one  set  of  achievable  goals  for  the 
organization  . . .  and  holding  the  most  senior  managers  accountable  for 
achievement  of  those  goals.”  Accordingly,  the  report  recommended  that 
DOT  expand  its  use  of  performance  agreements.  On  the  basis  of  these 
findings,  DOT  intends  to  continue  to  implement  performance  agreements 
at  lower  levels  and  consider  progress  towards  the  goals  in  those 
agreements  in  performance  evaluations.  For  fiscal  year  2000,  DOT 
instructed  the  heads  of  the  modal  administrations  to  enter  into 
performance  agreements  with  the  highest  level  members  of  the  Senior 
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Executive  Service  (SES)  in  their  administrations.  DOT  envisions  that  the 
agreements  will  eventually  cascade  through  the  entire  SES  ranks,  and  then 
below. 
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The  Office  of  Student  Financial  Assistance  (SFA),  which  administered 
more  than  $48  billion  in  student  financial  aid  to  more  than  8.5  million 
students  in  fiscal  year  1998,  has  been  on  our  high-risk  list  since  1992. 1  This 
has  been  due  in  part  to  the  number  of  SFAs  programs  that  have  been 
vulnerable  to  waste,  fraud,  abuse,  and/or  mismanagement. 

In  response  to  longstanding  management  weaknesses,  Congress  chartered 
SFA  as  a  Performance-Based  Organization  (PBO)  in  1998.  The  newly 
created  PBO — subject  to  the  authority  of  the  Secretary  of  Education — is 
responsible  for  administering  the  information  and  financial  systems  that 
support  SFA  programs,  excluding  the  development  of  policy,  and  exercises 
independent  control  of  its  budget  allocations  and  expenditures.  In 
chartering  SFA  as  a  PBO,  Congress  gave  SFA  personnel  authorities,  such  as 
the  ability  to  appoint  up  to  25  technical  and  professional  employees 
without  regard  to  the  provisions  of  title  5,  United  States  Code,  governing 
appointments  in  the  competitive  service.  Congress  also  provided  SFA  with 
certain  procurement  authorities,  such  as  the  ability  to  procure  property 
and  services,  subject  to  the  authority  of  the  Secretary  of  Education.  To  hold 
SFA  accountable  for  addressing  its  management  challenges  and  achieving 
results,  Congress  is  requiring  SFAs  Chief  Operating  Officer  (COO)  to  have  a 
performance  agreement  with  the  Secretary  of  Education.  SFA  also  is 
required  to  use  performance  agreements  between  the  COO  and  its  senior 
managers. 


Define  Accountability  for  The  development  of  the  performance  agreement  for  SFAs  COO  is  to  be  an 
Specific  Goals  annual  exercise  in  which  the  COO  and  the  Secretary  of  Education  agree  on 

^  measurable  organizational  and  individual  goals  that  the  COO  will  be 

accountable  for  achieving.  This  agreement,  once  completed,  is  to  be 
transmitted  to  Congress  and  made  publicly  available.  According  to  the 
legislation,  on  the  basis  of  progress  towards  the  goals  defined  in  his  or  her 
performance  agreement,  the  COO  can  receive  a  performance  bonus  of  up 
to  50  percent  of  his  or  her  basic  pay,  or  about  $61,000  in  2000.  The  COO  can 
also  be  removed  or  reappointed  on  the  basis  of  progress  towards  the  goals 
in  his  or  her  performance  agreement.  The  legislation  also  requires  the  COO 
to  enter  into  subsequent  performance  agreements  with  SFAs  senior 
managers.  The  agreements  are  parallel  to  the  agreement  between  the  COO 
and  the  Secretary.  A  senior  manager’s  salary  and  performance  bonus 


1  Major  Management  Challenges  and  Program  Risks:  Department  of  Education 
(GAO/OCG-99-5,  Jan.  1999). 
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cannot  exceed  125  percent  of  the  maximum  rate  of  basic  pay  for  the  Senior 
Executive  Service,  or  up  to  $153,000  in  2000.  A  senior  manager  can  be 
removed  at  the  discretion  of  the  COO. 

According  to  SFA,  its  performance  plan  serves  as  the  performance 
agreement  between  the  COO  and  the  Secretary  of  Education.  In  1999,  SFA 
published  its  5-year  performance  plan  that  sets  three  goals  for  SFA  and, 
consequently,  its  COO:  to  improve  customer  satisfaction,  improve 
employee  satisfaction,  and  lower  the  unit  cost  of  its  operations.  Appendix 
A  of  SFAs  performance  plan  sets  out  the  annual  performance  objectives 
and  corresponding  performance  measures  for  each  of  SFAs  five  senior 
managers:  General  Managers  for  Students,  Schools,  and  Financial  Partners; 
the  Chief  Financial  Officer  (CFO);  and  the  Chief  Information  Officer  (CIO). 
According  to  SFA,  this  appendix  also  serves  as  the  performance 
agreements  between  the  COO  and  the  senior  managers. 


Monitor  Progress  During  the  SFAs  approach  to  monitoring  progress  during  the  year  includes  reporting 
Year  of  progress  towards  the  goals  in  performance  agreements  and  meetings  to 

discuss  performance.  Progress  towards  the  goals  and  the  effects  of  internal 
or  external  factors  on  these  goals  are  discussed  weekly  by  two  groups.  The 
first,  SFAs  Management  Council,  comprises  the  COO;  the  Chief  of  Staff;  the 
CIO;  CFO;  and  the  General  Managers  for  Students,  Financial  Partners,  and 
Schools.  The  second,  the  Senior  Leadership  Team,  includes  the 
Management  Council  as  well  as  other  managers.  To  inform  these  meetings, 
SFA  produces  a  biweekly  performance  report,  which  details  SFAs  progress 
towards  its  performance  goals  and  highlights  goals  about  which  SFA  is 
concerned.  SFA  also  produces  quarterly  progress  reports,  which  show  its 
progress  in  meeting  organizational  goals. 


Contribute  to  Performance  SFAs  PBO  legislation  requires  it  to  prepare  and  submit  to  Congress  an 
Evaluations  annual  report  on  its  performance,  including  the  extent  to  which  it  met  the 

goals  in  its  performance  plan.  The  report  is  also  to  include,  among  other 
things,  the  evaluation  rating  for  the  COO  and  senior  managers,  including 
the  amounts  of  the  bonuses  they  received.  SFA  reported  that  it  met  most  of 
its  interim  goals  for  fiscal  year  1999  relating  to  improving  customer 
satisfaction,  reducing  the  overall  cost  of  student  aid,  and  transforming 
itself  to  a  PBO.  For  example,  SFA  reported  that  it  completed  its 
modernization  blueprint  and  met  its  goal  to  attract  3  million  electronic 
filings  of  student  aid  applications  by  the  end  of  the  fiscal  year.  On  the  basis 
of  SFAs  performance  in  fiscal  year  1999,  the  COO  received  a  performance 
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bonus  that  was  less  than  the  maximum  amount  allowed  under  the  PBO 
legislation.  According  to  officials,  SFA  based  the  amount  on  the  number  of 
interim  goals  SFA  met  in  1999. 

According  to  SFA  officials,  as  the  basis  to  evaluate  the  performance  of  its 
senior  managers,  SFA  plans  to  emphasize  that  senior  managers  are 
responsible  for  helping  SFA  to  accomplish  its  overall  goals,  as  well  as 
corresponding  goals  for  their  respective  organizations,  by  weighting  senior 
manager  performance  evaluations  accordingly.  Specifically,  progress 
towards  each  senior  manager’s  performance  goals  will  account  for  50 
percent  of  his  or  her  overall  evaluation.  The  remaining  50  percent  will  be 
determined  by  the  extent  to  which  SFA  reaches  its  three  overall 
performance  goals.  According  to  agency  officials,  SFA  managers  will 
receive  bonuses  based  on  this  evaluation. 

Though  SFA  is  in  the  early  stages  of  implementing  its  performance 
agreements,  officials  told  us  they  believe  the  concept  is  effective  and  worth 
replicating  at  lower  levels  of  the  agency.  Consistent  with  these  efforts, 
SFA’s  PBO  legislation  requires  it  to  establish  a  performance  management 
system  that  creates  goals  for  employee,  group,  or  organizational 
performance  that  are  consistent  with  the  PBO’s  performance  plan. 
According  to  SFA  officials,  it  has  begun  to  take  preliminary  steps  to 
develop  a  performance  management  system  to  meet  its  requirements. 


(410529) 
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The  first  copy  of  each  GAO  report  is  free.  Additional  copies  of 
reports  are  $2  each.  A  check  or  money  order  should  be  made  out  to 
the  Superintendent  of  Documents.  VISA  and  MasterCard  credit 
cards  are  accepted,  also. 

Orders  for  100  or  more  copies  to  be  mailed  to  a  single  address  are 
discounted  25  percent. 

Orders  by  mail: 

U.S.  General  Accounting  Office 
P.O.  Box  37050 
Washington,  DC  20013 

Orders  by  visiting: 

Room  1100 

700  4th  St.  NW  (comer  of  4th  and  G  Sts.  NW) 

U.S.  General  Accounting  Office 
Washington,  DC 

Orders  by  phone: 

(202)  512-6000 
fax:  (202)  512-6061 
TDD  (202)  512-2537 

Each  day,  GAO  issues  a  list  of  newly  available  reports  and 
testimony.  To  receive  facsimile  copies  of  the  daily  list  or  any  list 
from  the  past  30  days,  please  call  (202)  512-6000  using  a  touchtone 
phone.  A  recorded  menu  will  provide  information  on  how  to  obtain 
these  lists. 


Orders  by  Internet: 

For  information  on  how  to  access  GAO  reports  on  the  Internet, 
send  an  e-mail  message  with  “info”  in  the  body  to: 

info@www.gao.gov 

or  visit  GAO’s  World  Wide  Web  home  page  at: 


http://www.gao.gov 


To  Report  Fraud, 
Waste,  or  Abuse  in 
Federal  Programs 


Contact  one: 

•  Web  site:  http://www.gao.gov/fraudnet/fraudnet.htm 

•  e-mail:  fraudnet@gao.gov 

•  1-800-424-5454  (automated  answering  system) 
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